Image display apparatus and detection method

ABSTRACT

An image display apparatus is provided which includes an image display unit for displaying a still image or a moving image, an imaging unit for capturing an image in a direction in which the image display unit is displaying the still image or the moving image, a detecting unit for detecting a light emitting unit of a remote controlling device included in the image captured by the imaging unit, and detecting a movement amount of the light emitting unit within the captured image, the remote controlling device controlled by a user, and a control unit for controlling a movement amount of a pointer displayed on the image display unit, depending on the movement amount of the light emitting unit detected by the detecting unit.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. JP 2008-234508 filed in the Japanese Patent Office on Sep. 12, 2008,the entire content of which is incorporated herein by reference

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image display and a detectionmethod.

2. Description of the Related Art

Typically, a TV set merely generates image signals from airwavesreceived by an antenna to display images on the screen, whilst recently,TV sets capable of connecting to the Internet or the other networks havebeen developed and started to become popular. Such TV sets can connectto the Internet to display information via web browser, and in thiscase, it is convenience to use a pointing device which can indicate agiven point on the screen, as a mouse for use along with a personalcomputer.

For such a pointing device, a remote controller with a so-called “freecursor function” is proposed. This “free cursor function” enables toindicate a given point on the screen by moving the remote controller.The remote controller with the free cursor function can be picked up andmoved or waved round to indicate a given point on the screen or switchpages.

Also, as the functions and performances of TV sets andrecording/reproducing devices get more advanced, the functions which canbe controlled by a remote controller for remote controlling the TV setsand the recording/reproducing devices are increasing. Moreover, a remotecontroller with a display unit, such as a liquid-crystal display, hasbeen developed recently. The display unit of this remote controller maydisplay buttons that can be configured to run not only the basicfunctions, such as selecting the channels and adjusting the volume, butalso the functions of programmed recording and showing of a particularprogramme. The user may control the TV set by touching the display unit.

If the remote controller with the display unit provides the same buttonon the display unit for an inexperienced user in handling as one for anexperienced user, the inexperienced user is unable to utilise the remotecontroller well. It is thus desirable for such a controller to have afunction for customising for each user the matters to be displayed onthe display unit and allowing switching the matters for display for eachuser.

SUMMARY OF THE INVENTION

However, in the related art, a acceleration sensor and a gyro sensorneeds to be built in a remote controller in order to provide the freecursor function for the remote controller, which may result inundesirable increase of the production costs and power consumption ofthe remote controller.

Furthermore, in order to allow switching the matters for display of aremote controller with a display unit, such as a liquid-crystal display,there could be provided, for example, a method of preparing numberedbuttons on the remote controller which are to be pressed by the user, amethod of inputting an ID and an password into the remote controller,and a method of reading biological information, such as a fingerprintand the like, by the remote controller, each necessarily causing theuser to operate switching by himself, thereby resulting in complicatedoperations.

In light of the foregoing, it is desirable to provide an image displayapparatus and a detection method which are novel and improved, and whichallow adding readily a function to a remote controller by imaging theremote controller and the face of a human who operates the remotecontroller, and by detecting and identifying, from the captured image,the location of the remote controller and the face of the human whooperates the remote controller.

According to an embodiment of the present invention, there is providedan image display apparatus including an image display unit fordisplaying a still image or a moving image, an imaging unit forcapturing an image in a direction in which the image display unit isdisplaying the still image or the moving image, a detecting unit fordetecting a light emitting unit of a remote controlling device includedin the image captured by the imaging unit, and detecting a movementamount of the light emitting unit within the captured image, the remotecontrolling device controlled by a user, and a control unit forcontrolling a movement amount of a pointer displayed on the imagedisplay unit, depending on the movement amount of the light emittingunit detected by the detecting unit.

According to such a configuration, the image display unit displays astill image or a moving image, the imaging unit captures an image in adirection in which the image display unit is displaying the still imageor the moving image, the detecting unit detects a light emitting unit ofa remote controlling device included in the image captured by theimaging unit, and detects a movement amount of the light emitting unitwithin the captured image, and the remote controlling device iscontrolled by a user. Then the control unit controls a movement amountof a pointer displayed on the image display unit, depending on themovement amount of the light emitting unit detected by the detectingunit. As a result, the location of the emitting unit of the remotecontroller included in the image captured by the imaging unit, and themovement amount of the pointer indicating a given point on the imagedisplay unit is controlled depending on the detection result, so that afunction can be readily added to the remote controller.

The detecting unit may detect a face of a viewer included in the imagecaptured by the imaging unit, and detect the light emitting unit onlywithin an area in a predetermined distance from a location of thedetected face.

The detecting unit may detect a face of a viewer included in the imagecaptured by the imaging unit, and the controlling unit may locate thepointer at a centre of the image display unit if the detecting unitdetects that the light emitting unit is located at a position in apredetermined distance from the face detected by the detecting unit.

The control unit may control an emitting period of the light emittingunit to be in integer multiples of a capturing period of the imagingunit.

The control unit may change a light intensity of the light emitting unitif the detecting unit detects the light emitting unit.

The detecting unit may detect a face of a viewer included in the imagecaptured by the imaging unit, and the control unit may control anoperation of the remote controlling device by use of a detection resultfrom the detecting unit. In this case, the control unit may controlbutton display of the remote controlling device by use of the detectionresult from the detecting unit.

The control unit may estimate, by use of a detection result of the lightemitting unit detected by the detecting unit, a person who operates theremote controlling device if the detecting unit detects a plurality ofthe faces of the viewers.

The control unit may instruct the light emitting unit to emit light ifthe detecting unit detects the face of the viewer. Also, the controlunit may control the movement amount of the pointer, depending to adistance between the user and the imaging unit.

According to another embodiment of the present invention, there isprovided an detecting method comprising the steps of capturing an imagein a direction in which the image display unit that displays a stillimage or a moving image is displaying the still image or the movingimage, detecting a light emitting unit of a remote controlling deviceincluded in the image captured at the step of capturing, and detecting amovement amount of the light emitting unit within the captured image,the remote controlling device controlled by a user, and controlling amovement amount of a pointer displayed on the image display unit,depending on the movement amount of the light emitting unit detected atthe step of detecting.

According to the embodiments of the present invention described above,it is possible to provide an image display apparatus and a detectionmethod which are novel and improved, and which allow adding readily afunction to a remote controller by imaging the remote controller and aface of a human who operates the remote controller, and by detecting andidentifying, from the captured image, the location of the remotecontroller and the face of the human who operates the remote controller.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory view that illustrates the appearance of animage display apparatus 100 according to an embodiment of the presentinvention.

FIG. 2 is an explanatory view that shows the configuration of an imagedisplay apparatus 100 according to an embodiment of the presentinvention.

FIG. 3 is an explanatory view that illustrates the configuration of adetecting unit 106 included in an image display apparatus 100 accordingto an embodiment of the present invention.

FIG. 4 is a flow diagram that illustrates the operation of an imagedisplay apparatus 100 according to an embodiment of the presentinvention.

FIG. 5 is an explanatory view that illustrates a free cursor mode.

FIG. 6 is a flow diagram that illustrates the operation of an imagedisplay apparatus 100 according to an embodiment of the presentinvention.

FIG. 7 is an explanatory view that shows a case where buttons displayedby a remote controller 200 are changed in correspondence to a recogniseduser.

FIG. 8 is an explanatory view that shows a case where buttons displayedby a remote controller 200 are changed in correspondence to a recogniseduser.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Preferred embodiments of the present invention will be then describedhereinafter in the following order:

1. Overview of Image Display Apparatus According to Embodiment ofPresent Invention

2. Operation of Image Display Apparatus According to Embodiment ofPresent Invention

-   -   2-1. Realisation of free cursor function    -   2-2. Realisation of button display control of remote controller

3. Conclusion

1. OVERVIEW OF IMAGE DISPLAY APPARATUS ACCORDING TO EMBODIMENT OFPRESENT INVENTION

Firstly, an overview of an image display apparatus according to anembodiment of the present invention will be described. FIG. 1 is anexplanatory view that illustrates the appearance of an image displayapparatus 100 according to an embodiment of the present invention. Inthe following, the appearance of an image display apparatus according toan embodiment of the present invention will be described by use of FIG.1.

As shown in FIG. 1, the image display apparatus 100 according to anembodiment of the present invention includes, in the upper middle of adisplay panel 102 for images, an image input unit 104 for capturing amoving image. The image input unit 104 is configured to capture a movingimage in the direction in which the image display apparatus 100 displaysthe moving image by the display panel 102. The image display apparatus100 according this embodiment analyses the image captured by the imageinput unit 104 and detects a human's face in the image. The imagedisplay apparatus 100 is characterised by changing the internaloperation status of the image display apparatus 100 based on whether ahuman's face is included in the moving image captured by the image inputunit 104.

Besides, although the image display apparatus 100 in FIG. 1 includes, inthe upper middle of the display panel 102 for images, the image inputunit 104 for capturing a moving image, the present invention does not,of course, limit the location of an image input for capturing movingimages to the above example. For example, an apparatus which isdifferent from the image display apparatus 100 may be provided,connected to the image display apparatus 100, and may capture movingimages. Also, the number of image input units is not, of course, limitedto one, and more than one image input unit may be provided to captureimages.

In the above, an overview of the image display apparatus 100 accordingto an embodiment of the present invention has been described. Next, theconfiguration of the image display apparatus 100 according to anembodiment of the present invention

FIG. 2 is an explanatory view that shows the configuration of the imagedisplay apparatus 100 according to an embodiment of the presentinvention. In the following, the configuration of the image displayapparatus 100 according to an embodiment of the present invention willbe described by use of FIG. 2.

As shown in FIG. 2, an image display apparatus 100 includes a displaypanel 102, an image input unit 104, a detecting unit 106, a powercontrol unit 108, a SDRAM (Synchronous Dynamic Random Access Memory)110, a flash ROM 112, a CPU 114, a remote-controller-light receivingunit 116, a network I/F 118, a network terminal 120, a terrestrial wavetuner 122, a digital tuner 124, an audio A/D converting circuit 126, avideo decoder 128, an MPEG decoder 130, an audio signal processingcircuit 132, a picture signal processing circuit 134, an HDMI(High-Definition Multimedia Interface) receiver 136, an HDMI terminal138, an audio amplifying circuit 140, a speaker 142, a graphicsgenerating circuit 144, a panel driving circuit 146.

The terrestrial wave tuner 122 is configured to receive airwave signalstransmitted from a terrestrial wave antenna (not shown) for receivinganalogue terrestrial waves, and demodulate picture and audio signalsincluded in the airwave signals into baseband signals. The audio signalsof the baseband signals demodulated by the terrestrial wave tuner 122are transmitted to the audio A/D converting circuit 126, and the picturesignals of the demodulated baseband signals are transmitted to the videodecoder 128.

The digital tuner 124 is configured to receive airwave signalstransmitted from a digital antenna (not shown) for receiving digitalairwaves, convert the received airwave signals into an MPEG2-TS (MPEG2Transport Stream). Upon conversion of the airwave signals into theMPEG2-TS, the digital tuner 124 transmits the converted MPEG2-TS to theMPEG decoder 130.

The audio A/D converting circuit 126 is configured to receive the audiosignals demodulated by the terrestrial wave tuner 122, and convert theaudio signals from analogue into digital. Upon conversion of the audiosignals from analogue into digital, the audio A/D converting circuit 126transmits the converted digital audio signals to the audio signalprocessing circuit 132.

The video decoder 128 is configured to receive the picture signalsdemodulated by the terrestrial wave tuner 122, and convert the picturesignals from analogue signals into digital component signals. Uponconversion of the picture signals from analogue signals into digitalcomponent signals, the video decoder 128 transmits the converted digitalcomponent signals to the picture signal processing circuit 134.

The MPEG decoder 130 is configured to receive the MPEG2-TS transmittedfrom the digital tuner 124, and convert the audio from the MPEG2-TS intodigital signals, and the picture from the MPEG2-TS into digitalcomponent signals. Upon conversion from the MPEG2-TS into digital audiosignals and digital component signals, the MPEG decoder 130 transmitsthe converted digital audio signals to the audio signal processingcircuit 132, and the converted digital component signals to the picturesignal processing circuit 134.

The audio signal processing circuit 132 is configured to receive thedigital audio signals transmitted from the audio A/D converting circuit126 and MPEG decoder 130, and execute signal processing on the digitalaudio signals. Upon signal processing on the digital audio signals bythe audio signal processing circuit 132, the audio signals aretransmitted to the audio amplifying circuit 140.

The picture signal processing circuit 134 is configured to receive thedigital component signals transmitted from the video decoder 128 and theMPEG decoder 130, and execute signal processing on the digital componentsignals. Upon signal processing on the digital component signals by thepicture signal processing circuit 134, the digital component signals aretransmitted to the audio amplifying circuit 140.

The HDMI receiver 136 is configured to receive digital baseband signalsinput from the HDMI terminal 138 that is one of the external inputterminals. The digital baseband signals received by the HDMI receiver136 are transmitted to the audio signal processing circuit 132 and thepicture signal processing circuit 134, and signal processing is executedon audio signals and on digital component signals, respectively.

The audio amplifying circuit 140 is configured to receive the audiosignals output from the audio signal processing circuit 132, amplifythem by a predetermined amplification amount, and output the amplifiedsignals. The amount of the amplification by the audio amplifying circuit140 depends on the volume specified by the user of the image displayapparatus 100. The audio signals amplified by the audio amplifyingcircuit 140 are transmitted to the speaker 142. The speaker 142 outputsa sound based on the audio signals transmitted from the audio amplifyingcircuit 140.

The graphics generating circuit 144 is configured to generate graphicpictures that are necessary for the operation of the image displayapparatus 100. Different pictures are generated as the graphic picturesthat are necessary for the operation of the image display apparatus 100,according to the operation by the user. The graphic pictures generatedby the graphics generation circuit 144 are transmitted to the paneldriving circuit 146, multiplexed with or replaced by the picture signalstransmitted from the picture signal processing circuit 134. Besides, ifthe graphic pictures are not generated, the picture signals transmittedfrom the picture signal processing circuit 134 may be passed directly tothe panel driving circuit 146.

The panel driving circuit 146 is configured to generate, from thepicture signals transmitted from the graphics generating circuit, paneldriving signals that is necessary for displaying pictures on the displaypanel 102. The panel driving signals generated by the panel drivingcircuit 146 are transmitted to the display panel 102. The display panel102 operates depending on the panel driving signals, and pictures arethen displayed on the display panel 102.

The display panel 102 is configured to display moving images based onthe panel driving signals transmitted from the panel driving circuit146. In this embodiment, the display panel 102 display moving images byliquid crystals.

As described above, the image input unit 104 is provided in the uppermiddle of the display panel 102 for images, and configured to capture amoving image in the direction in which the image display apparatus 100displays the moving image by the display panel 102, when the paneldriving signals are supplied to the display panel 102 and the movingimage is displayed on the display panel 102. The image input unit 104may be configured to capture a moving image by a CCD (Charge CoupledDevice) or CMOS (Complementary Metal Oxide Semiconductor) image sensor.The moving image captured by the image input unit 104 is transmitted tothe detecting unit 106. Besides, the number of the image input unit 104may be one or more.

The detecting unit 106 is configured to get the moving images, which aretransmitted from the image input unit 104, input thereto, and detect ahuman's face included in the moving images and the location of thelight-emitting unit of the remote controller 200. The detected resultsfrom the detecting unit 106 of a human's face in the moving images andthe location of the light-emitting unit of the remote controller 200 aretransmitted to the power control unit 108, the CPU 114 and the like, andused for controlling the operation of the image display apparatus 100.Besides, the configuration of the detecting unit 106 will be describedin detail below.

The power control unit 108 is configured to control the powerconsumption of the image display apparatus 100 by controlling theoperation of each part of the image display apparatus 100. The operationof the power control unit 108 will be described in detail below, thoughfor now is explained with one example: when the detection by thedetecting unit 106 results in no human's face found in a moving imagethat is transmitted from the image input unit 104, the power controlunit 108 determines that nobody is watching the moving image displayedon the display panel 102 of the image display apparatus 100, and getscontrol to lower the brightness of the backlight of the display panel102.

Thus, depending on the situation of viewing moving images displayed bythe image display apparatus 100, the power consumption of the imagedisplay apparatus 100 can be controlled by controlling the operation ofeach part of the image display apparatus 100 by the power control unit108 depending on the detected results from the detecting unit 106.

The SDRAM 110 is a temporary work area for a computer program underexecution for controlling each part of the image display apparatus 100by the CPU 114. The flash ROM 112 is a ROM in which the computer programfor controlling each part of the image display apparatus 100 by the CPU114 is stored. And the CPU 114 is configured to read and executesequentially the computer program, which is stored in the flash ROM 112,for controlling each part of the image display apparatus 100.

The remote-controller-light receiving unit 116 is configured to receivesignals transmitted from the remote controller 200. The signals receivedby the remote-controller-light receiving unit 116 are input into the CPU114, by which the control codes included in the signals are decoded.Upon decoding the control codes, the CPU 114 controls each part of theimage display apparatus 100 to operate in response to the control codes(adjusting the volume, setting channels, displaying a menu picture,etc.).

The network I/F 118 is configured to receive signals input from thenetwork terminal 120 via a network, and transmit signals to the networkvia the network terminal 120. Upon receiving the signals input from thenetwork terminal 120 via the network, the network I/F 118 transmits thereceived signals to the CPU 114. The CPU 114 analyses the signalsreceived by network I/F 118, controls each part of the image displayapparatus 100 to operate in response to the signals.

Besides, each part of the image display apparatus 100 are connected byan internal bus 121, and configured to allow control of each part of theimage display apparatus 100 by the CPU 114. Also, the image displayapparatus 100 and the remote controller 200 may be in wirelesscommunication with each other. Thus, not shown in FIG. 2, a wirelesscommunication unit for wireless communication may be included in theimage display apparatus 100 and the remote controller 200.

In the above, the configuration of the image display apparatus 100according to an embodiment of the present invention has been described.Next, the configuration of the detecting unit 106 included in the imagedisplay apparatus 100 according to an embodiment of the presentinvention will be described.

FIG. 3 is an explanatory view that illustrates the configuration of thedetecting unit 106 included in the image display apparatus 100 accordingto an embodiment of the present invention. In the following, theconfiguration of the detecting unit 106 included in the image displayapparatus 100 according to an embodiment of the present invention willbe described by use of FIG. 3.

As shown in FIG. 3, the detecting unit that is included in the imagedisplay unit 100 according to an embodiment of the present inventionincludes a face detecting unit 152, a face recognising unit 154, and aremote-controller detecting unit 156.

The face detecting unit 152 is configured to execute a process (a facedetecting process) for detecting part of a human's face out from animage captured by the image input unit 104 and transmitted from theimage input unit 104. By detecting part of a human's face by the facedetecting unit 152, a control related to the human's face can beexecuted.

The face recognising unit 154 is configured to execute a further process(a face recognising process) for specifying a person upon the part of ahuman's face that is detected by the face detecting unit 152 out fromthe image captured by the image input unit 104 and transmitted from theimage input unit 104. By executing the face recognising process forspecifying a person by the face recognising unit 154, a control relatedto the specified person can be executed.

The remote-controller detecting unit 156 is configured to execute aremote-controller detecting process for detecting the location of theremote controller 200 out from an image captured by the image input unit104 and transmitted from the image input unit 104. By executing theremote-controller detecting process by the remote-controller detectingunit 156, a control related to the location of the remote controller 200can be executed.

The detecting unit 106 is thus configured, so that, out from an imagecaptured by the image input unit 104 and transmitted from the imageinput unit 104, a human's face can be detected and then the face can berecognised, and also the location of the remote controller 200 can bedetected. Further, using the detected results from the detecting unit106, controls related to the human's face, to the specified person, andto the location of the remote controller 200 can be executed within theimage display apparatus 100.

In the above, the configuration of a detecting unit 106 included in animage display apparatus 100 according to an embodiment of the presentinvention has been described by use of FIG. 3. Next, the operation of animage display apparatus 100 according to an embodiment of the presentinvention will be described.

2. OPERATION OF IMAGE DISPLAY APPARATUS ACCORDING TO EMBODIMENT OFPRESENT INVENTION 2-1. Realisation of Free Cursor Function

FIG. 4 is a flow diagram that illustrates the operation of the imagedisplay apparatus 100 according to an embodiment of the presentinvention. In the following, the operation of the image displayapparatus 100 according to an embodiment of the present invention willbe described by use of FIG. 4.

The flow diagram shown in FIG. 4 shows an operation in a case where itis detected whether a light emitting unit of the remote controller 200exists or not out from an image captured by the image input unit 104 andtransmitted from the image input unit 104, and where it is determinedwhether or not to switch to the free cursor mode, depending on theexistence of the light emitting unit of the remote controller 200.

Upon capturing a moving image by the image input unit 104, the movingimage captured by the image input unit 104 is analysed by the facedetecting unit 106, and the distance between the image display apparatus100 and a viewer is measured from the area taken by the face of theviewer out of the moving image captured by the image input unit 104.Then, as a result of the measurement, it is determined by the facedetecting unit 152 whether the measured distance is shorter than apredetermined distance (step S101). Besides, the distance between theimage display apparatus 100 and the viewer may not be preciselymeasured, but be measured in a rough range (for example, the location ofthe face exists at a distance of 2 m50 cm-3 m from the image input unit104).

If as a result of the determination of the face detecting unit 152 atstep S101, the measured distance is shorter than the predetermineddistance, any control related to the free cursor mode will not be taken,the operation then returns to step S101, and the determining process bythe face detecting unit 152 is repeated. On the other hand, as a resultof the determination of the face detecting unit 152 at step S101, if themeasured distance is longer than the predetermined distance, thensubsequently, it is detected whether the brightness in the room wherethe image display apparatus 100 is located reaches a predeterminedbrightness (step S102). It may be determined whether the brightness inthe room reaches the predetermined brightness, for example, as follows:given that the image input unit 104 is configured with a CCD imagesensor, an average for the image is obtained by the CCD image sensor.Then, it is determined by the CPU 114 whether the obtained average forthe image is equal to or above a predetermined value.

In this embodiment, if, as a result of the above-mentioned detection ofthe brightness in the room at step S102, it is determined that thebrightness in the room where the image display apparatus 100 is locateddoes not reach the predetermined brightness, then a switch to the freecursor mode will not be done. Of course, the present invention is notlimited to the above example, and a switching process to the free cursormode may be done even if the brightness in the room where the imagedisplay apparatus 100 is located does not reach the predeterminedbrightness. Also in the present invention, the above-mentioned distancedetecting process at step S101 and the above-mentioned process fordetecting the brightness in the room at step S102 themselves may beomitted.

Upon the process for detecting whether the brightness in the roomreaches the predetermined brightness at step S102, the image issubsequently input from the image input unit 104 into the detecting unit106 (step S103), a remote-controller detecting process is executed bythe remote-controller detecting unit 156 that is included in thedetecting unit 106. The remote-controller detecting process in thisembodiment is a process for detecting, from the image supplied by theimage input unit 104, the location of a infrared-ray emitting unit ofthe remote controller 200 operated by the user of the image displayapparatus 100. The image input unit 104 in this embodiment can image asa bright point the infrared-ray emitting unit of the remote controller200, which is invisible for humans. Then, the remote-controllerdetecting unit 156 detects the location of the infrared-ray emittingunit of the remote controller 200 imaged by the image input unit 104,and also the trajectory of the movement of the infrared-ray emittingunit of the remote controller 200 is tracked, so that the image displayapparatus 100 can switch to the free cursor mode.

As a result of the remote-controller detecting process by theremote-controller detecting unit 156, it is determined whether theremote-controller detecting unit 156 has successfully detected thelocation of the infrared-ray emitting unit of the remote controller 200(step S104). If, as a result of the detection at step S104, theremote-controller detecting unit 156 has unsuccessfully detected thelocation of the infrared-ray emitting unit of the remote controller 200,the operation returns to step S101, the determining process by the facedetecting unit 152 is repeated. On the other hand, if, as a result ofthe measurement at step S104, the remote-controller detecting unit 156has successfully detected the location of the infrared-ray emitting unitof the remote controller 200, the image display apparatus 100 iscontrolled to operate in the free cursor mode. For example, the CPU 114may control the image display apparatus 100 to operate in the freecursor mode.

When the image display apparatus is switched into the free cursor mode,the location of the infrared-ray emitting unit of the remote controller200 at given timing of reading is detected based on the image capturedby the image input unit 104. Then, the location of the infrared-rayemitting unit of the remote controller 200 at another timing of reading,so that the movement of the location of the infrared-ray emitting unitof the remote controller 200 can be figured out based on the differencein the locations of the infrared-ray emitting unit of the remotecontroller 200. Moving a cursor displayed on the display panel 102 basedon this movement amount of the location of the infrared-ray emittingunit enables a two-dimensional pointing, as in a mouse for use alongwith a personal computer (PC).

FIG. 5 is an explanatory view that illustrates a case where the imagedisplay apparatus 100 is switched into a free cursor mode, and where acursor displayed on the display panel 102 is moved by moving the remotecontroller 200 in hand. In FIG. 5, the image input unit 104 images theinfrared-ray unit 210 of the remote controller 200 in the user's hand,and the location of the infrared-ray emitting unit 210 included in theimage captured by the image input unit 104 is detected by theremote-controller detecting unit 156, so that the free cursor functionis realised. Furthermore, the cursor (not shown) displayed on thedisplay panel 102 can be also moved depending on the movement of theremote controller 200, by moving the remote controller 200 in the user'shand in various directions.

The movement amount of a cursor (not shown) displayed on the displaypanel 102, given when the user moves the remote controller 200 in hand,may be changed, for example, depending on the distance between the userand the image input unit 104. For example, the movement amount of thecursor given when the user moves the remote controller 200 in handbecomes smaller the longer the distance between the user and the imageinput unit 104 becomes. Also, the distance of the cursor displayed onthe display panel 102 may be allowed to change arbitrarily according tothe user's settings.

Besides, a result of the face detection from the face detecting processby the face detecting unit 152 may be used when the image displayapparatus 100 is operated in the free cursor mode. For example, when aface detecting process is done by the face detecting unit 152 to specifythe area of the face, a filtering may be performed to detect theinfrared-ray emitting unit 210 of the remote controller 200 only withina predetermined area in the right, left, and downwards directions fromthe specified face. For example, when a face detecting process is doneby the face detecting unit 152 to specify the area of the face, afiltering may be performed to detect the infrared-ray emitting unit 210of the remote controller 200 only within an area of 50 cm each in theright, left, and downwards direction from the specified face. Because ofsuch a filtering, the remote-controller detecting process are notnecessarily to be executed for the whole area within the image capturedby the image input unit 104, which can allow a certain realisation ofthe free cursor function with lighter processing.

Using a result of the face detection from the face detecting process bythe face detecting unit 152, the initial position (home position) of acursor and the centre position for the cursor are set for the case wherethe image display apparatus 100 is operated in the free cursor mode. Forexample, the cursor may be set to point at the centre of the displaypanel 102 when the infrared-ray emitting unit 210 of the remotecontroller 200 is located at 30 cm below the user's face and 20 cm rightto the user's face. With such a setting, the free cursor function by useof the remote controller 200 can be realised almost with satisfaction ofthe user's natural sense.

Furthermore, in order to detect more certainly the infrared-ray emittingunit 210 of the remote controller 200, infrared-ray emitting patterns ofthe infrared-ray emitting unit 210 may be configured to emit indifferent patterns from those for normal command transmissions of theremote controller. In that case, the infrared-ray may be configured towink in an emitting pattern that matches timing for reading of the imageinput unit 104 (for example, at a rate of timing in integer multiples ofthe frame rate for image capturing by the image input unit 104). Thus bycontrolling the emitting patterns, if, for example, the rate for imagecapturing by the image input unit 104 is 30 times per second, theinfrared-ray emitting unit 210 of the remote controller 200 is thenconfigured to wink at 15 Hz, so that the infrared-ray emitting unit 210of the remote controller 200 can be certainly extracted by deriving adifference to the last captured image, even if the noise components ofthe images obtained by the image input unit 104 are large. Moreover, theintensity of the emitted light in this case is desirably set larger thanthose for command transmissions of the remote controller, or setadaptively depending on a degree of recognising winks. Such setting ofthe intensity of the emitted light allows both certain locating of theposition of the light source and realising of a smaller powerconsumption.

Besides, timing of switching into the free cursor mode may be timingexplicitly instructed by the user of the image input apparatus 100, ortiming at which the image display apparatus 100 has started tocommunicate with an external network, such as the Internet etc., and atwhich, for example, a website on the Internet is displayed on thedisplay panel 102.

2-2. Realisation of Button Display Control of Remote Controller

FIG. 6 is a flow diagram that illustrates the operation of the imagedisplay apparatus 100 according to an embodiment of the presentinvention. In the following, the operation of the image displayapparatus 100 according to an embodiment of the present invention willbe described by use of FIG. 6.

The flow diagram shown in FIG. 6 shows a case where the image input unit104 images, the face of the user who controls the remote controller 200is then detected and recognised from the image transmitted from theimage input unit 104, and a user interface corresponding to therecognised user is displayed at the remote controller 200.

Upon capturing a moving image by the image input unit 104, the movingimage captured by the image input unit 104 is analysed by the facedetecting unit 106, and the distance between the image display apparatus100 and a human is measured from the area taken by the face of the humanout of the moving image captured by the image input unit 104. Then, as aresult of the measurement, it is determined by the face detecting unit152 whether the measured distance is shorter than a predetermineddistance (step S111). Besides, the distance between the image displayapparatus 100 and the human may not be precisely measured, but bemeasured in a rough range (for example, the location of the face existsat a distance of 2 m50 cm-3 m from the image input unit 104).

If, as a result of the determination of the face detecting unit 152 atstep S111, the measured distance is shorter than the predetermineddistance, any control related to the free cursor mode will not be taken,the operation returns to step S111, and the determining process by theface detecting unit 152 is repeated. On the other hand, as a result ofthe determination of the face detecting unit 152 at step S111, if themeasured distance is longer than the predetermined distance, thensubsequently, it is detected whether the brightness in the room wherethe image display apparatus 100 is located reaches a predeterminedbrightness (step S112). It may be determined whether the brightness inthe room reaches the predetermined brightness, for example, as follows:given that the image input unit 104 is configured with a CCD imagesensor, an average for the image is obtained by the CCD image sensor.Then, it is determined by the CPU 114 whether the obtained average forthe image is equal to or above a predetermined value.

In this embodiment, if, as a result of the above-mentioned detection ofthe brightness in the room at step S112, it is determined that thebrightness in the room where the image display apparatus 100 is locateddoes not reach the predetermined brightness, then no control over theuser interface for the remote controller 200 will be done. Of course,the present invention is not limited to the above example, and a controlover the user interface for the remote controller 200 may be done evenif the brightness in the room where the image display apparatus 100 islocated does not reach the predetermined brightness. Also in the presentinvention, the above-mentioned distance detecting process at step S111and the above-mentioned process for detecting the brightness in the roomat step S112 themselves may be omitted.

Upon the process for detecting whether the brightness in the roomreaches the predetermined brightness at step S112, the image issubsequently input from the image input unit 104 into the detecting unit106 (step S103), a face detecting process is executed by the facedetecting unit 152 that is included in the detecting unit 106, then aprocess for recognising the user's face is executed by the facerecognising unit 154, using the image of face obtained as a result ofthe face detection process.

For example, the techniques described in JP 2007065766 (A) and JP2005044330 (A) can be used in the face detecting process by the facedetecting unit 152 and in the face recognising process by the facerecognising unit 154. In the following, the face detecting process willbe described briefly.

In order to detect a human's face from an image, firstly, the locationof a face in a supplied image, the size of the face, and the directionof the face are detected respectively. By detecting the location andsize of the face, a face image, which is part of the image, can beclipped from the image. Characteristic parts of the face (facecharacteristics locations), such as characteristic parts like eyebrows,eyes, a nose, and a mouse, are detected based on the clipped face imageand information of the direction of the face. For the detection of theface characteristics location, characteristic locations can be detectedby applying a method that is called AAM (Active Appearance Models), forexample. AAM is described in T. F. Cootes, G. J. Edwards, and C. J.Taylor, “Active Appearance Models,” Proc. Fifth European Conf. ComputerVision, H. Burkhardt and B. Neumann, eds, vol. 2, pp. 484-498, 1998.

Upon detecting the face characteristic locations, a local characteristicquantity is calculated for each of the detected face characteristiclocations. Calculating the local characteristic quantities and keepingthe calculated local characteristic quantities stored along with theface image allow identification of a face from an image input by theimage input unit 104. For the method of detecting a face, the techniquesdescribed in JP 2007065766 (A) and JP 2005044330 (A) can be used, forexample, and therefore the details are not explained here. Also,according to a face image and face characteristic locations, it ispossible to determine from a human's face in a supplied image that thehuman is male or female and adult or child. Moreover, it is possible tospecify a person from an image input by the image input unit 104 bykeeping information of face characteristics locations and localcharacteristic quantities stored, and verifying, with the storedinformation, a result of the face detecting and recognising processes onthe image input by the image input unit 104.

As a result of inputting the image from the image input unit 104 to thedetecting unit 106 at above-mentioned step S113, and performing the facedetecting and recognising processes by the face detecting unit 152 andthe face recognising unit 154, respectively, it is then determinedwhether the human's face has been detected and recognised successfullyor unsuccessfully (step S114). If, as a result of the determination atstep S114, no human's face has been successfully detected or the userhas unsuccessfully recognised, the operation returns to step S111, andthe determining process by the face detecting unit 152 is repeated. Onthe other hand, if, as a result of the determination at step S114, ahuman's face has been successfully detected and the user has beensuccessfully recognised, it is determined whether the button informationof the remote controller 200 for the successfully recognised user isregistered in the image display apparatus 100 (step S115).

If, as a result of the determination at above-mentioned step S115, thebutton information of the remote controller 200 for the successfullyrecognised user is registered in the image display apparatus 100,buttons are displayed on the remote controller 200 based on theregistered button information (step S116). For example, in order todisplay buttons on the remote controller 200, a radio signal includingthe button information may be transmitted to the remote controller 200,and the radio signal may be analysed by the remote controller 200.

Here, an example of the way of registering button information of theremote controller 200 into the image display apparatus 100 is described.For example, in order to register button information of the remotecontroller 200 into the image display apparatus 100, the image inputunit 104 is caused to start capturing an image by manipulating withbuttons of the remote controller 200. A user who wants to registerbutton information of the remote controller 200 moves the user's faceinto the area that the image input unit 104 can image, and has itcaptured by the image input unit 104, and additionally, transmit fromthe remote controller 200 the button information that the user wants toregister. In the image display apparatus 100, the face recognisingprocess is performed by the face recognising unit 154 on the imagecaptured by the image input unit 104, and a result of the facerecognising process and the button information transmitted from theremote controller 200 are correlated and stored in the internal of theimage display apparatus 100 (e.g., the flash ROM 112). By thusregistering the button information of the remote controller 200 into theimage display apparatus 100, button information for each user can beobtained in correspondence to a result of imaging by the image inputunit 104. Besides, registering button information into the image displayapparatus 100 in the present invention is not, of course, limited to theabove example, and button information may be stored in the image displayapparatus 100, for example, by displaying information, such as“ADVANCED” and “SIMPLE,” on the remote controller 200 and the displaypanel, and selecting either “ADVANCED” or “SIMPLE.”

On the other hand, if, as a result of the determination atabove-mentioned step S115, the button information of the remotecontroller 200 for the successfully recognised user is not registered inthe image display apparatus 100, default buttons pre-registered in theremote controller 200 are displayed on the remote controller 200 (stepS117). For example, in order to display pre-registered default buttonson the remote controller 200, a radio signal including buttoninformation may be transmitted to the remote controller 200, and theradio signal may be analysed by the remote controller 200.

FIG. 7 and FIG. 8 are explanatory views that show cases where buttonsdisplayed by the remote controller 200 are changed in correspondence torecognised users. FIG. 7 shows a case where along with buttons forexecuting basic functions, such as channel selecting buttons and volumeadjusting buttons, other buttons for execute various functions are alsodisplayed on the remote controller 200. Examples of the other buttonsfor execute various functions may includes buttons for programmedrecording of programmes, buttons for programmed showing of programmes,buttons for playing, fast-forwarding, fast-backwarding, stopping, andpausing a recorded programme, and buttons for switching airwaves. On theother hand, FIG. 8 shows a case where only buttons for executing basicfunctions, such as channel selecting buttons and volume adjustingbuttons, are displayed on the remote controller 200. The displayedbuttons for executing basic functions, such as channel selecting buttonsand volume adjusting buttons, shown in FIG. 8 may be default buttonsthat is displayed when, as a result of the determination at step S115,the button information of the remote controller 200 for the successfullyrecognised user is not registered in the image display apparatus 100.

Thus, buttons displayed on the remote controller 200 can be changed bycapturing an image by the image input unit 104, detecting andrecognising a human's face in the captured image, and use a result ofthe recognition. Furthermore, when a user just simply sits in front ofthe image display apparatus 100, buttons corresponding to the user aredisplayed without any authentications of the user ID or any other IDnumbers and of finger-prints or any other biometrics. Therefore, anycomplicated input operations are no longer necessary, nor are includedany devices for biometric authentication in the remote controller 200.

Besides, if a plurality of human's faces are recognised by the facerecognising unit 154, an infrared-ray signal may be emitted from theinfrared-ray emitting unit 210 of the remote controller 200, thelocation of the infrared-ray emitting unit 210 of the remote controller200 may be detected by the remote-controller detecting unit 156, andthen button display of the remote controller 200 may be controlled byuse of the detected location of the infrared-ray emitting unit 210. Forexample, if a plurality of human's faces are recognised by the facerecognising unit 154, an infrared-ray signal may be emitted from theinfrared-ray emitting unit 210 of the remote controller 200, and thelocation of the infrared-ray emitting unit 210 of the remote controller200 may be detected by the remote-controller detecting unit 156. Then, ahuman whose face is the closest to the location of the remote controller200 may be estimated as the user of the remote controller 200, so thatbutton display of the remote controller 200 may be controlled. Besides,from the perspective of reducing power consumption, the operation foremitting an infrared-ray signal from the infrared-ray emitting unit 210of the remote controller 200, and detecting the location of theinfrared-ray emitting unit 210 of the remote controller 200 by theremote-controller detecting unit 154 is preferably limited to beperformed only when a plurality of human's faces are recognised by theface recognising unit 154.

In the above, the operation of the image display apparatus 100 accordingto an embodiment of the present invention has been described by use ofFIG. 6. Besides, the above-mentioned operation of the image displayapparatus 100 according to an embodiment of the present invention may beperformed by reading by the CPU 114 computer programs stored in theinternal of the image display apparatus 100 (e.g., ROM or any otherstorage means, which is not shown), and sequentially executing them tocontrol each part of the image display apparatus 100.

3. CONCLUSION

As described above, according to an embodiment of the present invention,the free cursor function or any other function can be realised in theremote controller 200, by capturing an image by the image input unit104, and detecting, from the captured image, the face of the user whooperates the remote controller 200 and the location of the infrared-rayemitting unit 210 of the remote controller 200. Furthermore, switchingof the user interface picture displayed on the remote controller 200 canbe controlled by the image display apparatus 100, by detecting andrecognising, from the captured image, the face of the user who operatesthe remote controller 200.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

For example, in the above-mentioned embodiment, the remote-controllerdetecting unit 156 detects the location of the remote controller 200 bydetecting an infrared-ray signal emitted by the infrared-ray emittingunit 210 of the remote controller 200, though the present invention isnot limited to the above example. For example, the remote controller 200may emit visible light, and the visible light emitted by the remotecontroller 200 may be detected so that the location of the remotecontroller 200 may be detected.

Also for example, the shape of the remote controller 200 may be imagedby the image input unit 104 to be pre-recognised by the image displayapparatus 100, so that the location of the remote controller 200 may bedetected by use of information of the recognised shape of the remotecontroller 200. Also for example, if there is anything like a cradle torest the remote controller 200 on, the location and shape of the cradlemay be pre-recognised by the image display apparatus 100, so that thelocation of the remote controller 200 may be detected by use ofinformation of the recognised location and shape of the cradle.Furthermore, the detecting unit 106 may detect, from the image capturedby the image input unit 104, the location of the user's arm with theremote controller 200, so that the location of the remote controller 200may be located.

Also for example, in the above-mentioned embodiment, the case has beendescribed where the user's face is recognised by the face recognisingunit 154, and where buttons displayed on the remote controller 200 arechanged by use of a result of the face recognition, though except forbuttons, pictures customised for the recognised user may be displayed onthe remote controller 200. For example, the contents viewing status ofthe recognised user may be analysed by the CPU 114, and informationrelated to the recommended contents the recognised user to view (forexample, titles, airtime, playback time, casts, and any otherinformation of the contents) may be displayed on the remote controller200 and the display panel 102. Furthermore, if the remote controller 200is capable for connecting to the Internet or any other externalnetworks, pages on the Internet which correspond to the recognised user(for example, pages set as Homes by the recognised user and pages foronline shopping, etc.) may be displayed on the remote controller 200 andthe display panel 102.

Also for example, the face detecting unit 152 and the face recognisingunit 154 may be configured not to consider a face included in an imagetransmitted from the image input unit 104 as human's, if the face staysat the same position for a predetermined time and won't move. By thusprocessing by the face detecting unit 152 and the face recognising unit154, no figures in a poster and a photograph will not falsely detectedand recognised.

What is clamed is:
 1. An image display apparatus comprising: an imagedisplay unit for displaying a still image or a moving image; an imagingunit for capturing an image in a direction in which the image displayunit is displaying the still image or the moving image; a detecting unitfor detecting an infrared light emitting unit of a remote controllingdevice included in the image captured by the imaging unit, and detectinga movement amount of the light emitting unit within the captured image,the remote controlling device controlled by a user; and a control unitfor controlling a movement amount of a pointer displayed on the imagedisplay unit, depending on the movement amount of the light emittingunit detected by the detecting unit, wherein the detecting unit detectsa face of a viewer included in the image captured by the imaging unitand uses a result of the face detection to detect the light emittingunit, the detecting unit using the result of the face detection byspecifying an area of the detected face and then applying a filteringsuch that the light emitting unit is detected only when the lightemitting unit is within the area of the detected face.
 2. The imagedisplay apparatus according to claim 1, wherein the detecting unitdetects a face of a viewer included in the image captured by the imagingunit, and the control unit locates the pointer at a centre of the imagedisplay unit if the detecting unit detects that the light emitting unitis located at a position in a predetermined distance from the facedetected by the detecting unit.
 3. The image display apparatus accordingto claim 2, wherein the control unit controls an emitting period of thelight emitting unit to be in integer multiples of a capturing period ofthe imaging unit.
 4. The image display apparatus according to claim 1,wherein the control unit changes a light intensity of the light emittingunit if the detecting unit detects the light emitting unit.
 5. The imagedisplay apparatus according to claim 1, wherein the detecting unitdetects a face of a viewer included in the image captured by the imagingunit, and the control unit controls an operation of the remotecontrolling device by use of a detection result from the detecting unit.6. The image display apparatus according to claim 5, wherein the controlunit controls button display of the remote controlling device by use ofthe detection result from the detecting unit.
 7. The image displayapparatus according to claim 5, wherein the control unit estimates, byuse of a detection result of the light emitting unit detected by thedetecting unit, a person who operates the remote controlling device ifthe detecting unit detects a plurality of the faces of the viewers. 8.The image display apparatus according to claim 5, wherein the controlunit instructs the light emitting unit to emit light if the detectingunit detects the face of the viewer.
 9. The image display apparatusaccording to claim 1, wherein the control unit controls the movementamount of the pointer, depending on a distance between the user and theimaging unit.
 10. A detecting method comprising the steps of: capturingan image in a direction in which the image display unit that displays astill image or a moving image is displaying the still image or themoving image; detecting an infrared light emitting unit of a remotecontrolling device included in the image captured at the step ofcapturing, and detecting a movement amount of the light emitting unitwithin the captured image, the remote controlling device controlled by auser; and controlling a movement amount of a pointer displayed on theimage display unit, depending on the movement amount of the lightemitting unit detected at the step of detecting, wherein the step ofdetecting comprises detecting a face of a viewer included in thecaptured image and using a result of the face detection to detect thelight emitting unit, the detecting unit using the result of the facedetection by specifying an area of the detected face and then applying afiltering such that the light emitting unit is detected only when thelight emitting unit is within the area of the detected face.